Passive internet measurement: Overview and guidelines based on experiences

Due to its versatility, flexibility and fast development, the modern Internet is far from being well understood in its entirety. A good way to learn more about how the Internet functions is to collect and analyze real Internet traffic. This paper addresses several major challenges of Internet traffic monitoring, which is a prerequisite for performing traffic analysis. The issues discussed will eventually appear when planning to conduct passive measurements on high-speed network connections, such as Internet backbone links. After giving a brief summary of general network measurement approaches, a detailed overview of different design options and important considerations for backbone measurements is given. The challenges are discussed in order of their chronological appearance: First, a number of legal and ethical issues have to be sorted out with legislators and network operators, followed by operational difficulties that need to be solved. Once these legal and operational obstacles have been overcome, a third challenge is given by various technical difficulties when actually measuring high-speed links. Technical issues range from handling the vast amounts of network data to timing and synchronization issues. Policies regarding public availability of network data need to be established once data is successfully collected. Finally, a successful Internet measurement project is described by addressing the aforementioned issues, providing concrete lessons learned based on experiences. As a result, the paper presents tutorial guidelines for setting up and performing passive Internet measurements.